Papers with unsupervised grammar induction
The Return of Lexical Dependencies: Neural Lexicalized PCFGs (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to grammar induction focus on discovering constituents or dependencies. |
| Approach: | They propose to model lexical dependencies using context free grammars instead of lexicals . they show that this unified framework induces both constituents and dependencies . |
| Outcome: | The proposed model overcomes sparsity problems and induces constituents and dependencies better than the current methods. |
Video-aided Unsupervised Grammar Induction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods of multi-modal grammar induction focus on grammar inducing from text-image pairs, but videos provide even richer information, such as static objects and actions. |
| Approach: | They propose a video-aided grammar induction model which learns a constituency parser from unlabeled text and its corresponding video. |
| Outcome: | The proposed model outperforms existing systems on three benchmarks. |
Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Chart-based models have shown great potential in unsupervised grammar induction, running recursively and hierarchically, but requiring O(n3) time-complexity. |
| Approach: | They propose a model-guided pruning method that scales to large language model pretraining by introducing a heuristic pruning method. |
| Outcome: | The proposed method significantly improves grammar induction quality and achieves competitive results in downstream tasks. |
Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction (P19-1)
Copied to clipboard
| Challenge: | In unsupervised grammar induction, data likelihood is only weakly correlated with parsing accuracy, especially at convergence after multiple runs. |
| Approach: | They propose to use VAS instead of data likelihood to find better grammars by examining linguistically-motivated constraints related to syntax. |
| Outcome: | The proposed model is better suited for word order typology classification than data likelihood. |
The Grammar of Emergent Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on emergent languages focus on semantics, but lack tools to analyse their properties. |
| Approach: | They propose to use unsupervised grammar induction techniques to analyse emergent languages and to examine their syntactic properties. |
| Outcome: | The proposed techniques are appropriate to analyse emergent languages and show that they exhibit syntactic properties similar to those observed in human language. |
Phrase-aware Unsupervised Constituency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have achieved inspiring success in unsupervised grammar induction using masked language modeling (MLM) as the proxy task. |
| Approach: | They propose to regularize the parser with phrases extracted by an unsupervised phrase tagger to help the LM model quickly manage low-level structures. |
| Outcome: | The proposed method improves the identification of high-level structures using phrase-guided masking. |
Structural Optimization Ambiguity and Simplicity Bias in Unsupervised Neural Grammar Induction (2024.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised grammar induction models lack analysis of traditional challenges, especially regarding the training loss. |
| Approach: | They propose a method to reduce the parse pool per sentence for loss evaluation using structural bias from pre-trained parsers. |
| Outcome: | The proposed method significantly improves performance while reducing variance and bias toward overly simplistic parses. |